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(54) Image processing apparatus, method and recording medium therefor 



(57) The invention intends, in the storage ol image 
information with a predetermined Image storage format, 
to add predetermined information to the input image 
matching such storage format. 

There is disclosed a configuration which generates 



plural image data of mutually different resolution levels, 
representing a same input image, and adding the pre- 
determined information to each of the images of the plu- 
ral resolution levels in a manner inconspicuous to the 
human eyes. 
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Description 

BACKGROUND OF THE INVENTION 

Field ot the Invention 

The present invention relates to an image process- 
ing apparatus for adding predetermined information to 
an input image, and a method and a storage medium 
therefor 

Related Background Art 

There is conventionally known a technology tor 
adding, to an input image, the manufacturing number of 
a copying apparatus which is used tor processing such 
image, as disclosed for example In the Japanese Patent 
Laid-open Application No. 5-244389. Such technology 
is effective in tracing the origin of the image from a re- 
produced image. 

On the other hand, there have recently been pro- 
posed various memory formats for storing image infor- 
mation. 

In such memory formats, however, there has not 
sufficiently been considered a method of adding certain 
information in a manner iuconspicuous or invisible to the 
human eyes. For this reason, it has not been possible, 
for example in case of storing plural image data files of 
mutually different resolution levels representing a same 
input image, to add the information associated with the 
image data in a suitable method. 

Also it has not been possible, for example In case 
of dividing the input image into plural areas and storing 
the image information in the unit of such divided area, 
to add the Information associated with the image infor- 
mation in a suitable method. 

Also in the conventional formats for image storage, 
in case of adding certain infomnation in a manner incon- 
spicuous or invisible to the human eyes, there has not 
sufficiently been considered the data format to be taken 
for the image including such added information. For this 
reason, the added information may be deteriorated or 
may become unreproducible for example by the influ- 
ence of subsequent data compression. 

SUIVIMARY OF THE INVENTION 

An object of the present invention is to provide an 
image processing apparatus, a method and a storage 
medium therefor, capable of individually or entirely solv- 
ing the drawbacks mentioned in the foregoing. 

Another object of the present invention is, under the 
above-mentioned object, to enable, in storing imago in- 
formation with a predetermined image storage format, 
the addition of predetermined information to the input 
image with a method suitable for such storage format. 

Still another object of the present invention is. under 
the above-mentioned object, to enable, in adding pre- 



determined information to the input image, the storage 
of image taking such addition of information into consid- 
eration. 

The above-mentioned objects can be attained, ac- 
5 cording to a preferred embodiment of the present inven- 
tion, by an image processing apparatus comprising: 

a division unit for dividing an input image into plural 
areas; 

10 a selection unit for selecting the storage format of 
the image data representing the input image in the 
unit of each of the divided areas: 
conversion means for converting the image data in- 
to the above-mentioned storage format: and 

15 addition means for adding predetermined informa- 
tion to the input image in the unit of each of the di- 
vided areas. 

Still another object of the present invention is to pro- 
20 vide an image processing apparatus, a method and a 
storage medium therefor, having a novel function or a 
novel mode of adding information to the given image in- 
formation. 

Still other objects of the present invention, and the 
25 foaturos thereof, will become fully apparent from the fol- 
lowing description of embodiments to be taken in con- 
junction with the attached drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

30 

Fig. 1 is a view showing the storage format embod- 
ying the present invention: 

Figs. 2 and 3 are views showing the configuration 
of an image file in the storage format embodying the 
35 present invention; 

Fig. 4 is a view showing an example of the image 
file composed of plural images of different resolu- 
tion levels; 

Fig. 5 is a view showing an example of tile division; 
40 Fig. 6 is a view showing an example of distribution 
of data formats in the tiles after tile division of the 
image; 

Fig. 7 Is a view showing an example of the hardware 
configuration embodying the present invention; 
45 Fig. 8 is a flow chart showing the procedure of file 
preparation; 

Fig. 9 is a flow chart showing the procedure of for- 
matting of image information; 

Fig. 10 is a flow chart showing the procedure of de- 
50 velopment into pixel data; 

Fig. 1 1 is a flow chart showing the procedure of color 
space conversion; 

Fig. 1 2 is a flow chart showing the procedure of res- 
olution level 0; 

55 Fig. 1 3 is a flow chart showing the procedure of a 
reduction process; 

Figs. 14 and 1 5 are flow charts showing the proce- 
dure of a watermark insertion process; 
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Fig. 16 is a flow chart showing the procedure of a 
process for each tile: 

Fig. 17 is a flow chart showing the procedure of a 
resolution level 0 process: 

Figs. 19, 19, 20. 21 and 22 are views showing ex- 5 
amples of watermark Insertion: 
Fig. 23 is a flow chart showing the procedure of a 
resolution level 0 process: 

Fig. 24 is a flowchart showing the procedure of tile 
attribute detection and data format determination: io 
and 

Fig. 25 is a flow chart showing the procedure of a 
process for each tile. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

Now the present invention will be clarified in detail 
by preferred embodiments thereof, with reference to the 
attached drawings. 

Al fiist there will be given an explanation on the stor- 
age format 

1 . File structure 

25 

(1) The storage format of the present embodiment 
is composed of objects of two types, namely a directory 
(shaded in Fig. 1 ) and a file (non-shaded). 

In an example shown in Fig. 1. a directory 1 consti- 
tutes a root directory, constituting the highest hierarchy 30 
in the storage format. The directory 1 includes two di- 
rectories (directories 2. 3) and a file (file 1 ). The directory 
2 includes an empty directory 4, and the directory 3 in- 
cludes two files (files 2, 3). 

The image information and the image attribute In- 3S 
formation are stored in a structure constituted by the 
above-mentioned two objects. 

(2) Configuration of image file 

40 

As shown in Fig. 2. each Image file (20) contains 
following components: 

(a) first attribute information (21 ) 

Summary information of the object image, in- 
eluding the format ID, title, producer, keyword, com- 
ment, final owner, revision number (number of times 
of savings of the object), total editing time, date and 
time of final print, date and time of original prepara- 
tion, date and time of final storage, thumbnail at- so 
tribute, and application used for preparation; 

(b) second attribute information (22) 

General information on the object of storage, 
including the heater, user type, clipboard format 
etc.: 

(c) third attribute information (23) 

Information on the output image Including the 
list of locked properties, image title after conversion. 



final editor, output image index, largest Image In- 
dex, largest conversion item index, largest opera- 
tion Index etc.: 

(d) Source Image object (24) 

It consists of image Information of the source 
image and attribute infornoation thereof, as will be 
explained later; 

(e) Image object after conversion (25) 

It consists of image information and attribute in- 
formation, obtained by applying a predetermined 
conversion process to the source image, and is 
structured similar to the source Image object; 

(f) fourth attribute information (26) 

Additional information of the original image or 
the image after conversion, including, on the source 
image object or on the converted image object, at 
least one or more of the Image ID, list of locked 
properties, title, final editor, revision number, date 
and time of preparation, date and time of final cor- 
rection, application used in preparation, status, 
number of conversions, list of conversion process- 
es, height/width of the contained image etc.; 

(g) fifth attribute information (27) 

Attribute information on the conversion of the 
source Image information, including at least one or 
more of the conversion item ID, operation class ID, 
list of locked properties, conversion title, final cor- 
rector, revision number, date and time of prepara- 
tion, date and time of final correction, application 
used for preparation, input data list, output data list, 
operation number, aspect ratio of process result, 
observed rectangular area, filtering geometric cal- 
culation, color conversion matrix, contrast adjust- 
ment etc.; and 

(h) sixth attribute information (28) 

Information for specifying the software for exe- 
cuting the conversion. 

(3) Configuration of Image object (30) 

The above-mentioned source image object (24) 
and converted image object (25) include following com- 
ponents: 

(a) seventh attribute information (31 ) 

Summary information of each object, including 
at least one or more of the fornnat ID, title, producer, 
keyword, comment, final owner, revision number 
(number of times of savings of the object), total ed- 
iting time, date and time of final print, date and time 
of original preparation, date and time of final stor- 
age, thumbnail image, thumbnail attribute, and ap- 
plication used for preparation; 

(b) eighth attribute information (32) 

General information on the object of storage, 
including the header, user type, clipboard format 
etc.; 

(c) ninth attribute information (33) 
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Attribute information on the method of storage 
of the image infornriRtion including the number of 
resolutions of the object image, width and height of 
the image of the maximum resolution, default width 
and height of the image display, unit of width and s 
height of the image display, width, height, color and 
format of the sub image, reducing method (method 
of preparation of the image of plural resolutions), 
width of the reduction prefilter, color processing pro- 
file of the sub image, compression parameters io 
(quantization table data of Huffman table data SOI, 
DOT, DHT, HOI in the JPEG compression) etc.: 

(d) tenth attribute information (34) 

Information associated with the image informa- 
tion, including the file source (original device of im- ^5 
age, scene type, image preparation history, soft- 
ware used tor preparation, user defined ID, sharp- 
ness), intellectual property (copyright indication, 
original image ownership, digital image ownership, 
producer, remarks), content description information 
(proofing data in image, description on image 
group, description on the title or object of image, de- 
scription on the person in the image, description on 
the object in the image, date of original image, de- 
scription on the event in the imago, description on ^5 
the location in the image, remarks), information on 
the camera used for image taking (camera maker 
name, camera model, camera manufacturing 
number), information on the setting of the camera 
at image taking (date and time of image taking, 30 
shutter speed, diaphragm stop, exposure program, 
luminance value (BV value), exposure correction, 
image taking distance, light metering mode, light 
source, focal length, fully open aperture value, 
flash, guide number, flush confirnnation, backlight- 35 
ing, position of object, frame number, special effect 
filter, remarks), digital camera characteristics (kind 
of image sensor. X-resolution on the focal plane, Y- 
resolutton on the focal plane, unit of resolution on 
the focal plane, spatial frequency characteristics, 40 
color filter array pattern, spectral sensitivity, ISO 
speed rating, OECF), film information (film brand, 
film category, film size, film roll number, film frame 
number), descriptive information on original docu- 
ment scanning in case the original is a book or a 45 
printed matter (original scanning image size, origi- 
nal document size, original medium, kind of origi- 
nal), scanning device information (scanner maker, 
scanner model, manufacturing number, scanner 
software, scanner software version number, scan- -so 
ning service company name, scanning person ID, 
scanning date, date and time of final proofing, scan- 
ner pixel size) etc., namely the information usable 
at the use of image such as how the image has been 
fetched and how the image can be utilized. 55 

(e) Image information 

It is composed, for a same image, image data 
(sub image data) 36. 39, 42 of plural resolution lev- 



els (35. 38, 41) and headers 37, 40, 43 of the sub 
image data of respective resolution levels. 

Instead of having the images of plural resolution lev- 
els, the file may also be composed of the image data of 
maximum resolution and the header thereof. 

The headers 37, 40, 43 contain the length of header 
stream, width and height of the image, number of tiles 
to be explained later, width and height of tile, number of 
channels, offset of tile header table, length of tile header 
entry, tile header table (consisting of tile offset, tile size, 
compression type and compression sub title). 

The tile offset indicates the position of the tile, while 
the time size indicates the amount of data in the tile, the 
compression type Indicates whether the tile data are 
non-compressed Image data, a single color, JPEG com- 
pressed image data or lack effective Image data, and 
the compression sub type indicates the color in case of 
singte-color data, or the interleaving type, chroma sub 
sampling method, presence/absence of internal color 
conversion and presence/absence of change in the 
JPEG table In case of JPEG compression. 

Fig. 4 shows an example of the Image file consisting 
of plural Images of different resolution levels, in Fig. 4, 
an imago 40 of the maximum resolution is composed of 
XO columns and YO rows, while an Image 41 of the sec- 
ond largest resolution Is composed of XO/2 columns x 
YO/2 rows. Thereafter the numbers of columns and rows 
are both reduced to 1/2 In successbn until those of col- 
umns and rows become respectively equal to or less 
than M and N, and the images are memorized down to 
an Image 42 of the minimum resolution. 

Because of such hierarchic structure, there are re- 
quired "number of hierarchic layers in an image file" In 
the attribute information of the Image, and the header 
Information and the Image Information for the Image In 
each hierarchic layer. The Informatbn on the number of 
hierarchic layers In an Image filed, the width and height 
of the Image of maximum resolution, the width, height, 
color configuration, compression method etc. of the Im- 
age of each resolution are described as the aforemen- 
tioned first attribute informatbn 21. 

As shown in Fig. 5, the Image of each resolution 
layer Is divided Into tiles each consisting of M x N pixels. 
When the image is divided into the tiles of fvl x N pixels 
starting from the upper left corner, a blank portion may 
be generated in a part of the tiles at the right-hand end 
and/or at the lower end (indicated by a hatched area). 

In such case the tile of f\/l x N pixels Is completed 
by Inserting the pixel at the right-hand end or at the lower 
end repeatedly. 

The Image of each tile Is memorized In a data format 
of JPEG compression, single color or non compression. 
The JPEG compression Is an image compressing meth- 
od of International standard defined by I SO/I EC 
JTC1/SC29. The single color is a data format, In case 
an entire tile is substantially composed of a single color, 
of representing the color of the tile by a single color in- 
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stead oi memorizing the individual pixel values. This da- 
ta format Is effective particularly for an image generated 
by computer graphiics. 

The Image data divided into the tile as explained in 
the foregoing are stored in the sub image data file, and 
the total number of tiles, the size of each tile, the data 
start position, the data format etc. are stored in the sub 
image header. 

Fig. 6 shows an example of the distribution of data 
formats of the tiles when the images 40, 41 , 42 in Fig. 
5 are divided into tile images 50, 51, 52. In Fig. 6, the 
attribute 1 indicates a non -compressed image, the at- 
tribute 2 indicates a JPEG compressed image and the 
attribute 3 indicates a single color 

Once the attribute is determined for each tile of the 
image ot a resolution level 0, the attribute of each of the 
images of the resolution levels 1 to n can be easily de- 
termined according to a predetermined rule. 

For example, in case all of the four tiles of the res- 
olution level 0. corresponding to a tile of the resolution 
level 1 .. have the attribute 1 (non-compressed image), 
such tile can be defined to have the attribute 1 . Also if 
all the four tiles have the attribute 3 (single color), the 
tile of the resolution level 1 can be defined to have the 
attribute 3. In case any of the four tiles has the attribute 
2, the tile of the resolution level 1 can be defined to have 
the attribute 2, and, if the four tiles have the attributes 1 
and 3, the tile of the resolution level 1 can be defined to 
have the attribute 1 . 

2. Hardware configuration 

An example of the hardware configuration in the 
present embodiment is shown in Fig. 7. 

There are shown a CPU 70 for executing the proc- 
ess sequence to be explained later; a RAM 71 used for 
example as a work area for the CPU 70: a ROM 72 stor- 
ing the programs to be executed by the CPU; an oper- 
ation unit 73 including a display unit and used as a user 
interlace; a hard disk 74 for storing data of the above- 
described data formats; a removable disk 75 such as a 
CD-R, a DVD-RAM or a floppy disk for storing data of 
the above -described data formats; an input interface 76 
consisting of a bidirectional interlace for entering image 
information and other information from an input device 
77 such as an image scanner, a digital camera, a digital 
video camera or the like; and an output interface 78 con- 
sisting of a bidirectional interface for outputling informa- 
tion to an external device through communication 
means such as a network 79. 

3. Preparation of image file 

Now there will be explained an example of the pro- 
cedure for preparing the image file described above. 

Fig. 8 shows the basic procedure executed by the 
CPU shown in Fig. 7. 

A step Si enters the image information from the in- 



put device 77 such as an image scanner, a digital cam- 
era or a digital video camera through the input interface 
76. 

A step S2 enters information associated with the im- 

5 age information and constituting the basis of the first to 
tenth attribute information described above. The entry 
is executed manually through the input interface 76 or 
the operation unit 73. 

A step S3 formats the image information, entered 

10 in the step 81 , Into the image file format of the above- 
described structure. 

A step S4 formats the attribute information, entered 
in the step S2, into the image file formal of the above- 
described structure. 

IS A step S5 stores the formatted image and attribute 
information in the hard disk 74 or the removable disk 75. 

In the following there will be explained, with refer- 
ence to Fig. 9, the formatting procedure of the image 
information in the step S3 in Fig. 8. 

20 A step S10 develops the entered image information 
into image data of respective pixels. The image data of 
respective pixels are prepared by an expansion process 
in case the image information is entered in a com- 
pressed format such as JPEG compression, or a devel- 

25 opmcnl process in case of entry in the PDL codes. Also 
in case the size of the entered image information has to 
be changed, the size conversion is achieved by trim- 
ming or zooming. This step is skipped if these processes 
are unnecessary. 

30 If the color space of the pixel data needs to be con- 
verted, a step S20 executes a color space conversion 
into any of predetermined color spaces such as RGB 
(additive primary colors) space and YCC (luminance 
and chromaticity) space. 

35 A step S30 executes formatting for the image of i 
maximum resolution, namely resolution level 0. as will 
be explained later in more details. The subsequent 
steps are not executed in case the image informatbn is 
formatted only in the maximum resolution level. 

40 A step S40 reduces the resolution to a half. The 
method of reduction is formatted as the ninth attribute 
described above. 

A step S50 executes formatting for thus prepared 
image of the resolution level 1. 

45 A step S60 executes a reduction process similar to 
that in the step S40. 

The reduction and the formatting are thereafter re- 
pealed in a similar manner, and a step S70 terminates 
the process by executing the formatting for the image of 

so the resolution level n. 

In the following there will be explained, with refer- 
ence to Fig. 10, the procedure of development into the 
pixel data, in the stop SIO in Fig. 9. 

A step S110 discriminates whether the entered im- 

55 age information is non -compressed pixel data, and the 
sequence jumps to a step S170 if the entered image in- 
formation is non -com pressed pixel data and does not 
require the development process. 



S 



9 



BP 0 892 545 A2 



10 



A step Si 20 discriminates whether the entered im- 
age information is encoded data, and. if so, a step SI 40 
executes decoding. 

A step Si 30 discriminates whether the entered im- 
age information is command data such as PDL, and^ if 
so, a step SI 50 executes development of the command. 

if the entered image information is data of any other 
fornnat and requires a process matching such format, a 
step SI 60 executes such process. 

A step 81 70 adjust the number of bits per pixel and 
the image size. 

In the following there will be explained, with refer- 
ence to Fig 1 1 , the procedure of color space conversion 
in the step S20 in Fig. 9. 

A step S210 presets a color space in manual man- 
ner through the operation unit 73. 

A step S220 discriminates whether the color space 
of the entered image information coincides with the color 
space set in the step 6210, and, in case of no coinci- 
dence, a step S230 executes the conversion of the color 
space. 

In the following there will be explained, with refer- 
ence to Fig. 12. the procedure for the image of the res- 
olution level 0 in the step S20 in Fig. 9. 

A stop S310 divides the image into tiles, each of M 
X N pixels. 

A step 8320 detects the attribute of the image of 
each of the divided tiles. For example a tile containing 
a character or a fine line is defined as a character/fine 
line tile; a tile containing relatively important information 
such as a human face is defined as an important tile; a 
tile of which all pixels are of same data (or if only a very 
limited number of pixels is different in value) is defined 
as a single color tile; and any other tile is defined as a 
halftone tile. 

A step S330 determines the data format of the tile 
according to the detected attribute. For example, the 
character/fine line tile and the important tile mentioned 
above are determined as the non-com press ion of the 
attribute 1 , the halftone tile is determined as the JPEG 
compression of the attribute 2, and the single color tile 
is determined as the single color of the attribute 3. 

A step S340 processes the image data of each tile 
so as to attain the determined data format. 

The foregoing procedure is basically applicable, in 
a similar manner, to the formatting of the images of the 
resolution level 1 to n. 

In the following there will be explained, with refer- 
ence to Fig. 1 3, the procedure of reduction in the step 
840 in Fig. 9. 

A step S410 presets the reduction method in man- 
ual manner through the operation unit 73. 

A step S420 oxccutGS reduction on the image data 
of the original resolution, by thus set reduction method. 

4. Insertion of watermark 

Now there will be explained the method of adding a 



watermark to the image in the present embodiment. 

The watermark means information related to the Im- 
age information of the above-described image file and 
so added to the image information as to be invisible or 

5 inconspicuous to the human eyes. The first to tenth at- 
tribute information mentioned in the foregoing may be 
inserted as the watermark mentioned above. 

An example of the method of inserting the water- 
mark will be explained in the following. 

10 ( 1 ) Fig. 1 4 shows the procedure of a watermark in- 
serting method 1, which inserts the watermark informa- 
tion into the object image in a real image space. 

A step S510 selectively reads the attribute informa- 
tion of the above-described file format, stored in the hard 

15 disk 74 or the removable disk 75 such as a CD-R. a 
DVD-RAM or a floppy disk and usable as the watermark 
information. 

A step 8520 effects enciphering on the extracted 
watermark information, and such enciphering can be 

20 achieved in various methods, such as a known conver- 
sion utilizing a random number table. Also the present 
invention is not limited by the enciphering method. 

A step 8530 converts the enciphered watermark in- 
formation, together with the pixels, into pattern data. 

25 A step 8540 utilizes the obtained image data of tho 

watermark information for mutual calculation with the 
image data of the object Image in the real image space, 
thereby embedding the watermark information in the ob- 
ject image. 

30 The mutual calculation can for example an addition 

or a multiplication. In any calculation, the amplitude (dy- 
namic range) of the watermark information is made suf- 
ficiently smaller than that of the object image, whereby 
the watermark information is added in inconspicuous 

35 manner to the human eyes. 

Also the color component of the watermark informa- 
tion to be added is selected as the B component of the 
RGB (additive primary color) data or the chromaticity 
component of the YCC (luminance and chromaticity) da- 

40 ta, so as that the delerioratbn in image quality is not 
easily recognized by the human eyes. 

(2) Fig. 1 5 shows the procedure of a watermark in- 
serting method II. which inserts the watermark informa- 
tion into the object image in the frequency space. 

45 A step 861 0 selectively reads the attribute informa- 
tion of the above-described file fornret, stored in the hard 
disk 74 or the removable disk 75 such as a CD-R, a 
DVD-RAM or a floppy disk. 

A step 8620 effects enciphering on the extracted 

so watermark information, and such enciphering can be 
achieved in various methods, such as a known conver- 
sion utilizing a random number table. 

A step 8630 converts the enciphered watermark in- 
formation, together with the pixels, into pattern data. 

55 A step 8640 converts the obtained image data of 

the watenmarl^ information into frequency components 
for example by orthogonal transformation. 

A step S650 converts the image data of the object 
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image into frequency components for example by or- 
thogonal transformation. 

A step S6S0 synthesizes the image data of the ob- 
ject image and the enciphered watermark information in 
the real innage space, thereby embedding the water- 
mark information into the object image. 

A step S670 executes inverse conversion of the 
synthesized image into the real Image data. 

(3) There wilt be explained, with reference to Fig. 
16, the procedure of above-explained watermark inser- 
tion into the Image data of the resolution level 1 . 

In the present embodiment, for inserting the water- 
mark, the process for the image data of each tile, in the 
step S340 in Fig. 12, is conducted in the following man- 
ner. 

A step SI 000 discriminates whether the data format 
is In the attribute 1 , and If so, a step SI 010 Inserts the 
watermark in the above-described method I. 

A step 81 020 discriminates whether the data format 
is In the attribute 2, and, if so, a step Si 030 inserts the 
watermark in the above-described method II, and a step 
81 040 executes block encoding. If the block encoding 
is conducted by a method employing orthogonal trans- 
formation such as the JPEG encoding, the inverse con- 
version in the step 8670 in Fig. 15 can bo omitted and 
the frequency components can be directly subjected to 
quantization and Huffman encoding. 

As explained In the foregoing, the watermark Insert- 
ing method is selected according to the memory format 
of the image data, whereby the watermark can be in- 
serted in a manner appropriate for each format. 

In case the step 81 020 Identifies that the data for- 
mat is not In the attribute 2. the tile is identified as in the 
attribute 3 and the representative value of the tile Is ex- 
tracted. In such case, the watermark is not inserted in 
the tile of this attribute. 

The foregoing process is repeated for all the tiles. 

The watermark Is inserted with the content and the 
method same as in the foregoing, also tor the images of 
the resolution levels 2 to (n-1 ). 

On the other hand, for the image of the resolution 
level n, the watermark is inserted In a procedure shown 
in Fig. 17. 

More specifically, as the image data of the resolu- 
tion level n can have only one attribute, the watermark 
is inserted by the above -described method I. 

As the watermark information is added in the 
present embodiment in the unit of a tile of M x N pixels, 
the watermark information is so selected that it is ac- 
commodated in a size of M X N pixels after enciphering 
and pattern development. 

Figs. 13 to 22 show examples of the content of the 
watermark Information to bo inserted and the arrange- 
ment thereof. 

[Example 1 of waternoark insertion] 

Fig. ie shows an example of the tile as In Fig. 6. 



wherein a indicates the information to be inserted as the 
waternnark. The content of the information a can be any 
or all of the aforementioned first to tenth attribute infor- 
matbn. for example the name of producer of the original 
s image and the date of preparation thereof. 

In the example shown in Fig. 18, the Information a 
is commonly Inserted in the tiles of all the hierarchic lay- 
ers. 

In this example, as the common information is in- 
10 serted In all the tiles of all the hierarchic layers, the name 
of the producer of the original image and the preparation 
date thereof can be obtained from the image of any layer 
so that the information source of the image data can be 
securely Identified. 

15 

[Example 2 of watermark insertionj 

Fig. 19 shows an example of the watermark inser- 
tion as in Fig. 16. 

20 in Fig. 1 9, a and b respectively Indicate information 
to be inserted as the watermark. The content of the in- 
formation a can be any or all of the aforementioned first 
to tenth attribute Information, preferably a part thereof, 
for example the name of producer of the original Image 

25 and the date of preparation thereof, while the content of 
the information b can be any or aJI of the aforementioned 
first to tenth attribute information, for example the name 
of the final storing person and the date and time of final 
storage. 

30 In the example shown in Fig. 19. the insertion of the 
information a or b Is switched according to the hierarchic 
layer. 

Such arrangement allows to increase the kind and 
the amount of the information to be Inserted, according 
35 to the increase in the number of hierarchic layers. 

[Example 3 of watermark insertion] 

Fig. 20 shows an example of the watermark inser- 

40 tlon as in Fig. 16. 

In Fig. 20, a and b respectively indicate information 
to be inserted as the watermark. The content of the In- 
formation a can be selected among the aforementioned 
first to tenth attribute information, for example the name 

45 of producer of the original image and the date of prep- 
aration thereof, while the content of the Information b 
can be, among the aforementioned first to tenth attribute 
Information, for example the name of the final storing 
person and the date and time of final storage. 

so In the example shown In Fig. 20, the Information a 
and b are switched according to the tile position within 
the same hierarchic layer. The information a and b are 
alternately inserted In such a manner that same Infor- 
mation Is not present In mutually adjacent tiles. 

55 Such arrangement of switching the information a 
and b according to the tile position within the same hi- 
erarchic layer allows to increase the kind and the 
amount of the information to be inserted, for a same 
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number of the hierarchic layers, and to obtain the infor- 
mation of plural kinds from the images of mutually adja- 
cent plural tiles. 

[Example 4 of watermark insertion] 

Fig. 21 shows an example of the watermark inser- 
tion as in Fig. 18. 

In Fig. 21, a, b, c, d, e and f respectively indicate 
information to be inserted as the watermark. The con- 
tent of the information a can be, among the aforemen- 
tioned first to tenth attribute information, for example the 
name of producer of the original image and the date of 
preparation thereof: the content of the information b can 
be for example the name of the final storing person and 
the date and time ot final storage, the content of the in- 
formation c can be for example the name of the appli- 
cation used in the preparation; the content of the infor- 
mation d can be tor example the keyword and the com- 
ments; the content of the information e can be for ex- 
ample the method of image information storage such as 
the compression parameters (in case of JPEG compres- 
sion, quantization table data or Huffman table data such 
as SOI, DQT, DHT, EOl etc.): and the content of the in- 
formation f can be for example the ID information of the 
device employed in the image input.. 

In the example shown in Fig. 21 , the information are 
switched according to the tile position with the same hi- 
erarchic layer, in such a manner that same Information 
is not present in mutually adjacent tiles. Also the kind 
and amount of the Inserted information are made larger 
for the higher resolution level and smaller for the lower 
resolution level. 

Such arrangement of switching the inserted infor- 
mation according to the tile position within the same hi- 
erarchic layer allows to increase the kind and the 
amount of the information to be inserted, for a same 
number of the hierarchic layers, and to obtain the infor- 
mation of plural kinds from the images of mutually adja- 
cent plural tiles. Also the kind and the amount of the in- 
serted infomnatlon can be made largerfor the higher res- 
olution level. 

[Example 5 of watermark insertion] 

Fig. 22 shows an example of the watermark inser- 
tion as in Fig. 18, wherein a indicates information to be 
inserted as the watermark. The content of the informa- 
tion a can be, among the aforementioned first to tenth 
attribute information^ for example the name of producer 
of the original image and the date of preparation thereof. 

In the example shown In Fig. 22, the information a 
is inserted into the tiles of a specified hierarchic layer. 

This example, In which the watermark is not insert- 
ed in all the layers but In a selected layer only, allows to 
insert the watermark in the Images of the important lay- 
ers, whereby the name of the producer of the original 
image and the preparation date hereof can be obtained 



from such image to securely identify the information 
source of the image data, while the process efficiency 
can be improved as the watermark insertion is not re- 
quired for other images. 

5 

[Variation 1] 

The foregoing embodiment employs plural water- 
mark inserting methods, but there may be employed on- 

10 ly one inserting method. _ 

More specifically, all the watermarks may be insert- 
ed by the inserting method I or II. The use of a common 
watermark inserting method allows to simplify the soft- 
ware or the hardware for watermark insertion. 

15 Also the intensity of watermark insertion may be 
made different in different layers, even by employing a 
same inserting method. 

For example, the watermark inserted in the image 
data of a high resolution level is so formed, in the mutual 

20 calculation with the image data of the object image, that 
the pattern data of the watermark information assume 
relatively snriall values in order to prevent the deteriora- 
tion of the image quality, and the watermark inserted in 
the Image data of a low resolution level is so formed, in 

25 the mutual calculation with tho image data of the object 
image, that the pattern data of the watermark informa- 
tion assume relatively large values in order to prevent 
the deterioration of the watermark information. 

Such watermark inserting method, selected accord- 

30 ing to the hierarchic level of the image data, allows to 
insert the Intensity matching the resolution level of the 
image data. 

[Variation 2] 

35 

Also the watermark inserting method may be 
switched according to the resolution level. 

For example, there may be employed the aforemen- 
tioned inserting method I for the image of the resolution 
40 level 0, and the inserting method II for the inreges of 
other levels. Such method allows to promptly insert the 
watermark with a relatively simple algorithm for the im- 
age of a high resolution level with a large data amount. 

45 [Variation 3] 

In the foregoing embodiments, the watermark is in- 
serted in all the tiles according lo the memory format o! 
the image data, but it is also possible to insert the wa- 
50 termark in a part of the tiles of the object image, by pe- 
riodically or randomly selecting the tiles in which the wa- 
termark is Inserted. 

Such arrangement allows to suppress the deterio- 
ration of the image quality resulting from the watermark 
55 insertion. 

As explained in the foregoing, the embodiments of 
the present invention allow, in storing the image infor- 
mation with a predetermined image storage format, to 
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add predetermined information to the input image with 
a method matching the storage format 

In the following there will be explained still another 
embodiment of the present invention, which is shown in 
Fig. 23 as a variation of the embodiment shown in Fig. 
12, 

Fig. 23 shows the procedure of the present embod- 
iment on the image of the resolution level 0 in the step 
S20 shown in Fig. 9. 

A step Si 310 inserts the watermark into the object 
image, in a method to be explained later. 

A step Si 320 divides the image into tiles, each hav- 
ing a size of M x N pixels. 

A step SI 330 detects the attribute of the image of 
the divided tile. 

A step S 1 340 determines the data format according 
to the delected attribute. 

A step SI 350 processes the image data of each tile 
so as to assume the determined data format. 

The procedure described above can also be basi- 
cally applied in a similar manner to the formatting of the 
images of the resolution levels 1 to no. 

Now reference is made to Fig. 24 for explaining the 
detection of the attribute of the tile and the determination 
of the data format. 

A step S710 discriminates whether th© watermark 
is inserted in the tile, and, if inserted, the data format is 
defined as the attribute 1 (step S720). 

A step S730 defines an important tile if the tile con- 
tains a character, a fine line or a relatively important in- 
formation such as a human face, and the data format is 
defined as the non-compression attribute 1 (step 8720). 

A step S740 defines a single color tile if all pixels of 
the tile are of same data (or if only a very limited number 
of pixels is different in value) and the data format is de- 
fined as the single color attribute 3 (step S760). Any oth- 
er tile is defined as a halftone tile, with the JPEG com- 
pression attribute 2 (step 5750). 

In the following there will be explained, with refer- 
ence to Fig. 25, the procedure of inserting the watermark 
into the image data of the resolution level 0. 

In the present embodiment, the process on the im- 
age data of each tile in the step 8340 in Fig. 23 is con- 
ducted in the following manner to the watermark inser- 
tion. 

A step S2000 discriminates whether the data format 
is in the attribute 1 , and, if so, the process is skipped as 
the image data are not compressed. 

A step 820 1 0 discriminates whether the data format 
is in the attribute 2, and, if so, a step 82020 executes 
block encoding. If the block encoding employs orthogo- 
nal transformation and quantization as in the JPEG en- 
coding, the imago is deteriorated by the non-revorsiblo 
encoding. 

If the step 82020 identifies that the data format is 
not in the attribute 2, the data format is identified as in 
the attribute 3 and the representative value of the tile is 
extracted (step 82030). 



The foregoing process is repeated for all the tiles. 

The above-described method can also be applied 
to the images of the resolution levels 1 to n. 

As the watermark information is added in the 
5 present embodiment in the unit of a tile of M x N pixels, 
the watermark information is so selected that it is ac- 
commodated in a size of M x N pixels after enciphering 
and pattern development. 



In the foregoing embodiment the encoding is not ex- 
ecuted on the tile in which the watermark is inserted, but 
it is also possible to adopt encoding in so-called lossless 
15 encoding manner which does not result in the deterio- 
ration of the image quality. More specifically, the control 
means for encoding is so designed as to select a pre- 
determined encoding method in case predetermined in- 
formation is added. 
20 Also, even for a same encoding method, there may 
be selected such an encoding parameter as to reduce 
the image quality deterioration by encoding. For exam- 
ple the deterioration of the image quality can be sup- 
pressed by controlling the quantizing parameter. More 
25 specifically, the control means for encoding may be so 
designed as to select a predetermined.encoding param- 
eter in case predetermined information is added. 

The present invention may be applied to a part of a 
system consisting of plural equipment (for example host 
30 computer, interface devices, reader, printer etc.) or a 
part of an apparatus consisting of a single equipment 
(such as a copying machine or a facsimile apparatus). 

Also the present invention is not limited to the ap- 
paratus or the method for realizing the aforementioned 
35 embodiments but also includes a case where program 
codes of a software for realizing the aforementioned em- 
bodiments are supplied to a computer (CPU or MPU) in 
the above-mentioned system or apparatus and the func- 
tions of the aforementioned embodiments are realized 
40 by the computer of the above-mentioned system or ap- 
paratus, which causes various devices to function ac- 
cording to such program codes. 

In such case the program codes themselves of the 
software realize the functions of the aforementioned 
45 embodiments, and such program codes themselves 
and means for supplying the computer with such pro- 
gram codes, namely a storage medium storing the pro- 
gram codes are also included in the present invention. 
The storage medium storing such program codes 
50 can be, for example, a floppy disk, a hard disk, an optical 
disk, a magnetooptical disk, a CD-ROM, a magnetic 
tape, a non volatile memory card or a ROM. 

The present invention also includes such program 
codes, not only in the case where the functions of the 
55 aforementioned embodiments are realized by the con- 
trol of the above-mentioned computer on various devic- 
es according to the supplied program codes but also in 
a case where the program codes realize the aforemen- 
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tioned embodiments in cooperation with an operating 
system functioning on the computer or another applica- 
tion software. 

The present Invention further includes a case 
wherein the supplied program codes are once stored in 
a function expansion board of the computer or a memory 
provided in a function expansion unit connected to the 
computer and a CPU or the like provided in the function 
expansion board or the function expansion unit exe- 
cutes all the process or a part thereof according to the 
instructions of such program codes, thereby realizing 
the functions of the aforementioned embodiments. 

Though the image formats shown in Figs. 2 to 5 are 
adopted in the foregoing embodiments, the present in- 
vention is not limited to such image formats but can also 
employ other image formats. 

As explained in the foregoing, embodiments of the 
present Invention can achieve, in adding predetermined 
information to the input image, image storage taking 
such addition of information into consideration. 



Claims 

1. An image processing apparatus comprising: 

division means for dividing an input image-into 
plural areas; 

control means for controlling the storage format 
of image data representing the input image in 
the unit of the divided area: 
conversion means for converting said image 
data into said storage format: and 
addition means for adding predetermined infor- 
mation to said input image in the unit of said 
area. 

2. An image processing apparatus according to claim 
1, wherein said addition means Is adapted to add 
the predetermined information with a method vari- 
able according to the storage fornnat of said image 
data. 

3. An image processing apparatus according to claim 
1, wherein said addition means is adapted to add 
the predetermined information with an intensity var- 
iable according to the storage format of said image 
data. 

4. An image processing apparatus according to claim 
1, wherein said addition means is adapted to add 
the predetermined information to the image data of 
a predeterminod storage format within said imago 
data. 

5. An image processing apparatus according to claim 
1, wherein said addition means is adapted to add 
said predetermined information in a completed form 



within said area. 

6. An image processing apparatus according to claim 
1 , wherein said addition means is adapted to add 

5 the predetermined information to the image data of 

a predetermined color component within said image 
data. 

7. An image processing apparatus according to claim 
10 1 , wherein said addition means is adapted to add 

said predetermined information in a manner incon- 
spicuous to the human eyes. 

8. An image processing apparatus comprising: 

75 

division means for dividing an input image into 
plural areas; 

storage means for storing the Input image in the 
unit of the divided area: and 
20 addition means for adding predetermined infor- 

nriation to said input image in the unit of said 
area. 

9. An image processing apparatus according to claim 
25 8, wherein said addition means is adapted to add 

the predetermined information with a method vari- 
able according to the storage format of the input im- 
age by said storage means. 

30 10. An image processing apparatus according to claim 
8, wherein said addition means is adapted to add 
the predetermined information with an intensity var- 
iable according to the storage format of the input 
image by said storage means. 

35 

11. An image processing apparatus according to claim 
8, wherein said addition means is adapted to add 
the predetermined information to the image data of 
a predetermined storage format within said input 

40 image. 

12. An image processing apparatus according to claim 
8, wherein said addition means is adapted to add 
said predetermined information in a completed form 

4S within said area. 

13. An image processing apparatus according to claim 
8, wherein said addition means is adapted to add 
the predetermined information to the image data of 

so a predetermined color component within the image 
data representing said input image. 

14. An image processing apparatus according to claim 
8, wherein said addition means is adapted to add 

55 said predetermined Information in a manner incon- 
spicuous to the human eyes. 

15. An image processing method comprising steps of: 
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dividing an input image into plural areas: 
controlling the storage format of image data 
representing the input image in the unit of the 
divided area: 

converting said image data into said storage 
format; and 

adding predetermined information to said input 
image in the unit of said area. 

16. An image processing method comprising steps of: 

dividing an input image into plural areas: 
storing the input image in the unit of the divided 
area: and 

adding predetermined information to said input 
image in the unit of said area. 

17. A computer readable storage medium storing a pro- 
gram for executing an image processing method for 
adding predetermined information to an input Im- 
age, the program comprising: 

a division step of dividing an input image into 
plural areas: 

a selection step of selecting the storage format 
of the image data representing the input image, 
in the unit of the divided area; 
a conversion step of converting said image data 
into said storage format; and 
an addition step of adding predetermined infor- 
mation to said input image in the unit of said 
area. 

18. A computer storage medium storing a program for 
executing an image processing method for adding 
predetermined information to an input image, the 
program comprising: 

a division step of dividing an input image into 
plural areas; 

a storage step of storing the input image in the 
unit of the divided area; and 
an addition step ol adding predetermined infor- 
mation to said input image in the unit of said 
area. 

19. An image processing apparatus comprising: 

generation means for generating plural image 
data of mutually different resolution levels, rep- 
resenting a same input image; and 
hiding means for hiding predetermined infor- 
mation in each of the images of said plural res- 
olution-levels in a manner inconspicuous to the 
human eyes. 

20. An image processing apparatus according to claim 
19, wherein said hiding means is adapted to hide 



said predetermined information in an algorithm var- 
iable according to the resolution level of said image 
data. 

21. An image processing apparatus according to claim 
19, wtierein said hiding means is adapted to hide 
said predetermined information in an intensity var- 
iable according to the resolution level ol said image 
data. 



JO 



22. An image processing apparatus according to claim 
19, wherein said hiding means is adapted to select 
whether or not to hide said predetermined informa- 
tion according to the resolution level of said image 

IS data. 

23. An image processing apparatus according to claim 
19, wherein said hiding means is adapted to select 
the content of said predetermined information ac- 

20 cording to the resolution level of said image data. 

24. An image processing apparatus according to claim 
19, wherein said hiding means is adapted to control 
the amount of said predetermined information ac- 

25 cording to the resolution level of said imago-data. 

25. An image processing apparatus according to claim 
19. wherein said hiding means is adapted to add 
predetermined information to the image data of a 

30 predetermined color component among said image 
data. 

26. An image processing method comprising steps of: 



35 



40 



generating plural image data of mutually differ- 
ent resolution levels, representing a same input 
image; and 

hiding predetermined information in each of the 
images of said plural resolution levels in a man- 
ner inconspicuous to the human eyes. 



27. A computer readable storage medium storing a pro- 
gram for executing an image processing method for 
adding predetermined information to an input im- 
45 age. the program comprising: 

a generation step of generating plural image 
data of mutually different resolution levels, rep- 
resenting a same input image; and 
so a hiding step of adding predetermined informa- 

tion in each of the innages of said plural resolu- 
tion levels in a manner inconspicuous to the hu- 
man eyes. 

55 28. An image processing apparatus comprising; 

addition means for adding predetermined infor- 
mation to an input image; 
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encoding means for encoding said input image; 
and 

control means for controlling the encoding of 
said input image according to the addition state 
of said predetermined information. 

29. An image processing apparatus according to claim 
28, wherein said addition means is adapted to add 
said predetermined information in the unit of a pre- 
determined area. 

30. An image processing apparatus according to claim 
28, wherein said addition means is adapted to add 
said predetermined information in a completed form 
within said area. 

31. An image processing apparatus according to claim 
28, wherein said addition means is adapted to add 
the predetermined information to the image data of 
a predetermined color component within said image 
data. 

32. An image processing apparatus according to claim 
28, wherein said addition means Is adapted to add 
said predetermined information in a manner incon- 
spicuous to the human eyes. 

33. An image processing apparatus according to claim 
28, wherein said control means is adapted to select 
a predetermined encoding method by said encod- 
ing means in case said predetermined information 
is added by said addition means. 

34. An image processing apparatus according to claim 
33, wherein said predetermined encoding method 
is a lossless encoding. 

35. An image processing apparatus according to claim 
28, wherein said control means is adapted to select 
a predetermined encoding parameter by said en- 
coding means in case said predetermined informa- 
tion is added by said addition means. 

36. An image processing apparatus according to claim 
35, wherein said predetermined encoding parame- 
ter is an encoding parameter involving little deteri- 
oration in the image quality in tfie encoding. 

37. An image processing apparatus according to claim 
23, wherein said control means is adapted not to 
execute the encoding by said encoding means in 
case said predetermined information is added by 
said addition means. 

38. An image processing apparatus according to claim 
28, wherein said control means is adapted to control 
the encoding by said encoding means in the unit of 
a predetermined block. 



39. An image processing method comprising steps of: 

adding predetermined information to an input 
image: 

5 encoding said input image: and 

controlling the encoding of said input image ac- 
cording to the addition state of said predeter- 
mined information. 

10 40. An image processing apparatus comprising: 

divisbn means for dividing an input image into 
plural areas; 

storage control means for storing the input im- 
15 age in the unit of the area divided by said divi- 

sion means: and 

addition means for adding predetermined infor- 
mation to said input image in the unit of said 
area. 

20 

41 . An image processing apparatus according to claim 
40, wherein said addition means is adapted to add 
said predetermined information in a completed form 
within said area. 

25 

42. An image processing apparatus according to claim 
40, wherein said addrtion means is adapted to add 
the predetermined information to the image data of 
a predetermined color component within said image 

30 data. 

43. An image processing apparatus according to claim 
40, wherein said addition means is adapted to add 
said predetermined information in a manner incon- 

35 spicuous to the human eyes. 

44. An image processing method comprising steps of: 

dividing an input image into plural areas; 
40 storing the input image in the unit of the divided 

area; and 

adding predetermined information to said input 
image in the unit of said area. 

45 45. A computer readable storage medium storing a pro- 
gram for executing an image processing method of 
adding predetermined information to an input im- 
age, the program comprising: 

so an addition step of adding predetermined infor- 

mation to an input image; 
an encoding step of encoding said input image; 
and 

a control step of controlling the encoding of said 
55 input inrrage according to the addition state of 

said predetermined information. 

46. A computer readable storage medium storing a pro- 
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gram for executing an image processing nnethod for 
adding predeternnined information to an input im- 
age, the program comprising: 

a division step of dividing an input image into 5 
plural areas: 

a storage control step of storing the input image 
in the unit of the area divided by said division 
means; and 

an addition step of adding predetermined infor- io 
mation to said input image in the unit of said 
area 
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